Minimum Bayes-Risk Decoding cons for Information Retrie
نویسندگان
چکیده
The paper addresses a new evaluation measure of automatic speech recognition (ASR) and a decoding strategy oriented for speech-based information retrieval (IR). Although word error rate (WER), which treats all words in a uniform manner, has been widely used as an evaluation measure of ASR, significance of words are different in speech understanding or IR. In this paper, we define a new ASR evaluation measure, namely, weighted word error rate (WWER) that gives a weight on errors from a viewpoint of IR. Then, we formulate a decoding method to minimize WWER based on Minimum BayesRisk (MBR) framework, and show that the decoding method improves WWER and IR accuracy.
منابع مشابه
Minimum Bayes-Risk Decoding for Statistical Machine Translation
We present Minimum Bayes-Risk (MBR) decoding for statistical machine translation. This statistical approach aims to minimize expected loss of translation errors under loss functions that measure translation performance. We describe a hierarchy of loss functions that incorporate different levels of linguistic information from word strings, word-to-word alignments from an MT system, and syntactic...
متن کاملRisk based lattice cutting for segmental minimum Bayes-risk decoding
Minimum Bayes Risk (MBR) decoders improve upon MAP decoders by directly optimizing loss function of interest: Word Error Rate MBR decoding is expensive when the search spaces are large Segmental MBR (SMBR) decoding breaks the single utterance-level MBR decoder into a sequence of simpler search problems. – To do this, the N-best lists or lattices need to be segmented We present: A new lattice se...
متن کاملEfficient Path Counting Transducers for Minimum Bayes-Risk Decoding of Statistical Machine Translation Lattices
This paper presents an efficient implementation of linearised lattice minimum Bayes-risk decoding using weighted finite state transducers. We introduce transducers to efficiently count lattice paths containing n-grams and use these to gather the required statistics. We show that these procedures can be implemented exactly through simple transformations of word sequences to sequences of n-grams....
متن کاملLater-stage Minimum Bayes-Risk Decoding for Neural Machine Translation
For extended periods of time, sequence generation models rely on beam search as the decoding algorithm. However, the performance of beam search degrades when the model is over-confident about a suboptimal prediction. In this work, we enhance beam search by performing minimum Bayes-risk (MBR) decoding for some extra steps at a later stage. In our experiments, we found that the conventional MBR r...
متن کاملGeneralized Minimum Bayes Risk System Combination
Minimum Bayes Risk (MBR) has been used as a decision rule for both singlesystem decoding and system combination in machine translation. For system combination, we argue that common MBR implementations are actually not correct, since probabilities in the hypothesis space cannot be reliably estimated. These implementations achieve the effect of consensus decoding (which may be beneficial in its o...
متن کامل